A Bayesian approach to sparse dynamic network identification

Modeling and identification for high dimensional (i.e. signals with many components) data sets poses severe challenges to off-the-shelf techniques for system identification. This is particularly so when relatively small data sets, as compared to the number signal components, have to be used. It is often the case that each component of the measured signal can be described in terms of few other measured variables and these dependence can be encoded in a graphical way via so called “Dynamic Bayesian Networks”. Finding the interconnection structure as well as the dynamic models can be posed as a system identification problem which involves variables selection. While this variable selection could be performed via standard selection techniques, computational complexity may however be a critical issue, being combinatorial in the number of inputs and outputs. Parametric estimation techniques which result in sparse models have nowadays become very popular and include, among others, the well known Lasso, LAR and their “grouped” versions Group Lasso and Group LAR. In this paper we introduce two new nonparametric techniques which borrow ideas from a recently introduced Kernel estimator called “stable-spline” as well as from sparsity inducing priors which use `1-type penalties. Numerical experiments regarding estimation of large scale sparse (ARMAX) models show that this technique provides a definite advantage over a group LAR algorithm and state-of-the-art parametric identification techniques based on prediction error minimization.